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Abstract 

We formulate the rudiments of a method for assessing the difficulty of dividing 
a computational problem into "independent simpler parts." This work illustrates 
measures of complexity which attempt to capture the distinction between "local" 
and "global" computational problems. One such measure is the covering multi- 
plicity, or average number of partial computations which take account of a given 
piece of data. Another measure reflects the intuitive notion of a "highly inter- 
connected" computational problem, for which subsets of the data cannot be 
processed "in isolation." These ideas are applied in the setting of computational 
geometry to show that the connectivity predicate has unbounded covering multi- 
plicity and is highly" interconnected; and in the setting of numerical computations 
to measure the complexity of evaluating polynomials and solving systems of 
linear equations. 

This is a preprint of a paper to appear in The Journal of Theoretical Computer 
Science. 
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Towards a Theory of Local and Global in Computation 
Harold Abelson 

1 . Introduction 

Many approaches to computational complexity focus on issues concerning the speed of 
computation: How many basic operations are required to compute a given function? How can 
computation time be decreased by performing operations in parallel rather than serially? 
What are the time-space tradeoffs for a given class of algorithms?; There'is, however, another 
kind of complexity. This is the organizational, or structural complexity of processes realized 
by large numbers of interconnected elements. Structural complexity is an important concern 
in many of the difficult areas to which computational methods are just beginning to be applied. 
For example, in exploring computational models for vision [6], one is struck by the fact that 
for a contemporary digital computer, the ratio, of connections to components is about three, 
whereas for the mammalian cortex it lies between 10 and 10,000.' A comparison, such as this 
raises a challenge for theoretical computer science: Is it possible to characterize those 
computational problems whose solution is inherently better suited to a highly interconnected 
structure than to a weakly interconnected structure? 

A similar issue arises in the study of distributed computation and computer networks. 
In this setting, an entity such as a data base might be widely distributed among the nodes of a 
network. Analyses of sorting or searching the data base must be concerned not only with 
elementary operations but also with internodal communication. Consider, for example, solving 
a large system of linear equations, where each column of the matrix resides at a different 
node. If the network is far-flung, the time and cost of communication could dominate the 
solution process, and arithmetic operations performed at individual nodes might be viewed 
simply as overhead. In a case like this, one would be concerned with minimizing, not 
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necessarily the number of operations, but rather the total amount of information which must 
be shipped across the network. What complexity measure is appropriate to this problem? 

Many of the issues in structural complexity revolve around the notion of "local and 
global" or "parts and wholes." Loosely speaking, a computational problem is inherently local if 
it can be divided into small, weakjy interacting modules. A computational problem is inherently 
global if any way of dividing it into pieces must entail substantial interaction among the 
pieces. Creating a useful theory of local and global is of course a formidable task, and this 
paper can be no more than an initial attempt. I introduce a measure, called the covering 
multiplicity, which reflects the organizational complexity of a problem in the sense hinted at 
above. Covering multiplicity is, roughly, the number of independent parts of a process which 
must take account of a given piece of data. In visual processing this might be, for example, 
the average number of "low level" elements influenced by a given patch of the retina. The 
concept of covering multiplicity syrely does not. capture all of what might be meant by "local 
and global in computation;" but it is at feast a start 

This introduction continues with a review of the setting established by Minsky and 

Papert [7] in their analysis of the perceptnon. We will make use of the same basic framework, 

although many of Minsky and Papert's techniques, relying fundamentally on the linearity of the 

perceptron's decision element, are unsuitable in the present, more general setting. Section 2 

of the paper begins the formal presentation, providing definitions both of covering multiplicity 

and also another complexity measure based on the idea of a "highly interconnected" 

computation. We find that computations which determine whether or not a geometric figure is 

connected must exhibit arbitrarily high covering multiplicity, and must be highly 

interconnected, thus providing a justification Of Minsky and Papert's intuitive guess [7]: 

^^ "We chose [in studying perceptrons] to investigate connectedness because 

of a belief that this predicate is nonlocal in some very deep sense." 

Section 3 turns from geometry to the computation of real-valued functions and gives a 



■J^^Hir 



Local and Global in Computation — September 20, 1977 — 3 



necessary and sufficient condition for "computational decomposability" which is used to 
identify multivariate polynomials whose evaluation requires arbitrarily high covering 
multiplicity. We also discuss the matrix problem cited above. The conclusion notes some of 
the many questions which are left untouched in this initial treatment of local and global 
complexity. 

LL The Perceptron framework 

Minsky and Papert's theory begins with an idealized retina R, which is simply a 
collection of n points. Figures on the retina are subsets IcR, We can think of R as the 
squares in a two-dimensional plane grid and -arbitrary geometric figures" as approximated by 
some collection of squares. A predicate on R is a function / from figures on R to {0, J}. The 
support of / is the set of alt points of R which affect the value of /, and the order of / is the 
size of support(/). A perceptron is a predicate which has the form 

; f{X) - T I, a.f.(X) > $ 1 
where / f are predicates and fi, a [f a 2 , .-. . are real numbers. (We follow Minsky and Papert in 
using the notation Tsome condition! to signify the predicate whose value is J if the condition is 
true and if the condition is false.) The order of the perceptron is the maximum order of any 
of the /.. Minsky and Papert characterize the complexity of geometric predicates in terms of 
the order of the perceptrons which compute them. They demonstrate, for example, that the 
predicate TX is connected! has infinite order, that is, the order of any perceptron which 
computes, connectivity must become arbitrarily large as the size of the retina becomes large. 
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1*2. Insufficiency of the perccptron analysis as a general theory of local and global 

The notion of order has considerable appeal as a characterization of "local versus 
global": A low order perceptron is local — the partial predicates can make independent 
computations based on small patches of the retina. A high order computation is global ~ 
individual partial predicates must access portions of the retina. Unfortunately, this 
characterization fails when we consider structures more general than the perceptron. 

A perceptron can be viewed as a composition of functions / - g(f^ f^ . • ^ f r ) where 
g is a predicate on Boolean r-tuples. In a perceptron, g must be a linear threshold function. 
We would tike to consider more general computational schemes in which there are no 
restrictions on g. Extending Minsky and Papert's results to this more general setting, 
however, raises many problems. Consider their paradigm result: Connectivity is not of finite 
order. This follows from an analysis whose main step is: Parity t that is, the predicate 

TX contains an even number of points! 
is not of finite order* But this can be true only in the linear threshold context. If we omit this 
restriction, then the determination of parity can be as "local* as we please: For any arbitrary 
division of the retina into disjoint sets S* let 

/.(X) -TX H S- has even parity! '. • 
Then X has even parity if and only if the product over all i of 2/.(X) - I is positive. 

To obtain a hold on what makes the above parity computation "local" and why we 
suspect that any connectivity computation must be "global" notice that for parity the supports 
are disjoint — each point of the retina is examined by only one partial predicate. Moreover, 
any division of the retina into disjoint sets can serve as the partial predicate supports for 
computing parity. Let's examine in this light a predicate which we might agree is "local," the 
property of being locally convex, which can be be determined by checking that X has 
non-negative curvature at each of its boundary points. Even though this determination is 
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"local," it cannot be easily realized with disjoint supports. Suppose, for example, that we 
divide the retina in two disjoint halves and attempt to compute local convexity. Consider the 
shape in Figure 1. Although it is not convex, the point of negative curvature will be 
undetected in either support. But this is hardly a fundamental problem. We merely need to 
allow a bit of overlap as in Figure 2, so that a few points of the retina lie in more than one 
support, and our local computation can proceed without problem. We suspect, though, that no 
such simple scheme can work for connectivity. In determining connectivity, we would guess, 
points must in general be accessed by many partial predicates. This provides motivation for 
our definition of covering multiplicity: 

The covering multiplicity fi i$ the average number of partial predicates 

which examine a given point on the retina* 

2. Support structures for geometric predicates 

2.L Basic definitions 

Definition 2.1.1. For any function of n variables fix^ . . ., * R ) the set Jt • {1, . . ., n) is called 

the retina of /. The support of / is the smallest set S C R such that 

fix ^ . . ., * n ) • fiyp • . ., y n ) 
whenever x* - y. for all i € S. 

Throughout Section 2, we shall assume that the functions concerned are Boolean functions of 
Boolean variables, although later on we shall also consider real-valued functions of real 
variables. Note that if R is finite, then "any function has a unique support. 
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Figure 1 
With disjoint supports the point of negative 
curvature is missed 




Figure 2 
Curvature is detected using overlapping suppctts 
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Definition 2.1.2. A support structure on a retina J? is a sequence of the form 

« - is v . . ., s r > 

where Sj<zR. Note that the S • need not be distinct. The number r is called the rank of IL 
The order of H is the maximum oyer / of |5 ,|. For any A c R define cov^(A) to be the 
number of supports of the structure which intersect A: 

covjjwj-i^iinsyy 0}\ 

The covering multiplicity of It is the average over ft of the number of supports containing a 
given point: 

li{H) - (lln) 2J |€|? cpv<{l}> 

Note that the sum over f? of cov({i}) is precisely the number of pairs (i, S •) where i € S *. 
Therefore, if all the 5 .-have the same size k, then the covering multiplicity, the order, the 
rank and the size of the retina are related by u(H)n - kr , 

Definition 2.1.3 A support structure H « ($}} is said to admit a function f(xp . . ., x ) if / can 
be represented. in the form 

/ - ldf v . • -, f r ) 
where support(/p « Sj for * - I, . . ., r. 

If / is a function from {0, !} R to {0, 1}, i.e., a "predicate on R," we might try to define 
the covering multiplicity of / to be the minimum covering multiplicity of any support structure 
which admits /. But this will not work, since any predicate is admitted by the multiplicity I 
structure ({i}, {2} f . . ., [n]). Covering multiplicity, therefore, is not a useful measure of 
complexity when considering structures consisting of many small supports. We will 
concentrate on the opposite situation, in which we attempt to keep the ranks of the structures 
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bounded for large retinas by using larger and larger supports. One example of this kind of 
structure is the fractional support structure^ in which each support is some fixed fraction of 
the entire retina: 

Definition 2.1.4. Let Af be a positive integer. A support structure ffona retina jR is. said to be 
a 1 1 M- fractional support structure if each support in H has size rt/Af. 

(IS Af does not divide n evenly we suppose each support to have size within ±1/2 of this 
value.). 

Strictly speaking, of course, a predicate /is defined only for a particular retina, so it 
makes no formal sense to speak of "computing / On large retinas." On the other hand, we can 
think of properties like "parity" and "connectedness" as defining entire families of predicates, 
one predicate for each retina. We can now define the covering multiplicity of such a predicate 
(family) in the context of fractional supports. 

Definition 2.1.5. For any predicate / on R and positive integer Af we define j*(/, Af, /?), the 
covering multiplicity of f for 1/M- fractional supports to be the minimum p of any 
1/Af -fractional support structure which admits /. 
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Definition 2.1.6. A predicate family / is said to have covering multiplicity at most B for all 
fractional supports, ji(/; frac) < B, if, for all M, /4(/, Af, R) is uniformly bounded by B on large 
retinas. That is, for any M there should exist a bound n^ such that ji(/, M, R) < B for any 
retina R with \R\ > n^. If such a finite value B exists, we ?ay that / has finite covering 
multiplicity for fractional supports. 

2*2. Examples 

To illustrate the above definitions, we compute the covering multiplicity for 
l/M -fractional supports for the local convexity predicate, using the the computation outlined 
in Section 1.2. We noted thet local convexity can be determined by examining each boundary 
point of the figure in question, and that this can be done with supports that do not overlap, 
.except for small "interfaces" along the edges. 

Suppose that the retina is a square h units on a side., n - h . Using supports of order 
n/M we divide R into M strips, each strip a rectangle of si?e h by h/M. We must also assign 
additional partial functions to examine the interfaces between the strips. (See Figure 3.) Each 
interface is itself a $trip of width 2, and thus one additional support (of order h [M) can 
contain h/2M interfaces {each of size 2h\ Since there are Af-I interfaces in all, we need 
2M(M-l)fh additional "interface supports." So the entire structure for local convexity has 
r « rank - *(strips) + ^{interface supports) « M * 2M(M-l)fh 
k - order - h 2 /M 
p -fcr/ii - 1 + 2{M-l)lh - I + 2(M-l)Wn 
Thus we have shown 
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Figure 3 
Retina divided into strips 
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Example 2.2.1. On a square retina R, predicate / - TX is locally convex! has 

/i(f,tf,R)< I + 2(M-I)/Vn 
Consequently, for any c > 0, we have p(f; frac) < I + V. 

As a second example, consider the predicate TX is a single* solid rectangle!. This can 
be computed by partial functions which -count the corners of X." That is, a figure X is a single 
solid rectangle if and only if its boundary contains precisely four "convex corners" (Figure 4a), 
no "concave corners" (Fig. 4b) and no "double corners" (Fig. 4c). This computation can be 
performed with almost the same support structure as used above, i.e., by dividing the retina 
into disjoint strips together with "interface" strips of width I. This time, however, we assign 
three partial functions, hence three supports, to each of the strips. In each support A 9 . the 
three corresponding partial functions /,, f^ f$ output or i as follows: 

L If X PI A contains concave corners, double corners, or more than four convex 
corners, then all f. output I. 

2. Otherwise the three functions output the number (from to 4, counting in binary) of 
convex corners of X f) A. 

Using this information, the function g can determine whether or not X is .a rectangle. The 
covering multiplicity here is three times as great as for local convexity: 

Example 2.2.2. If / is the predicate TX is a single solid rectangle! then for any « > 0, 
lk(f% frac) < 3 ■+ «. 

These examples illustrate predicates that can be computed with small covering 
multiplicity. On the other hand, we find that no fixed bound on covering multiplicity can 
suffice for computing arbitrary predicates on large retinas: 
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Figure 4 
Corner clusters for detecting rectangles 
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Proposition 2.2.3. Let M be a positive integer, and € a positive real number. Then any 
1 1 M- fractional support structure H which admits all predicates on a retina R must have 

M (H) > (it - t)/M 
provided r is sufficiently large with respect to M. 

«rt 
Proof. There are 2 predicates on a retina of size n. Consider, on the other hand, the number 

of predicates admitted by support structures of rank r and order ft. For each of the r partial 

predicates /. there are f " ) ways of choosing support{/p; and having selected a support, 

9 k 2 r 

there are then 2* functions /. with that support. In addition, there are there are 2* 



possibilities for-*. So if the structure admits any function of n variables, we must have 

Using the fact that ( n j < 2 n f and taking logarithms, gives 

2 n <rn + f ±r2 k 
or, since ft - n/M and r - n^/ft - Mp 

2 n < 2 M * i + Miiin + 2 nlM ) D 



2J$. Highly interconnected computations 

Covering multiplicity is a measure of the "globainess" of a computation, which can 
provide meaningful results about families of supports in which the order grows large along 
with the size of the retina. For arbitrary supports we can consider instead the question of 
how much "interconnection" -among elements of the retina is required for a given computation. 
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Definition 23.1. If H « ( Sj, ... ., S f ) is a support structure and A c R define Conjj(yJ) c R to 
be the set 

Con w (/l) - U { S. : S. fl /I ^ } 

Intuitively, imagine that H is "wired** by connecting together all pairs of elements in each 
support. Then Con{A) consists of all those points of R with "direct connections* to points of 
A. By way of analogy with 2.2.3, we will show that any computational scheme which can 
compute arbitrary Boolean functions must be "highly interconnected" in the sense that, for any 
A C R, either A must intersect many supports or else Con{A) must contain essentially all of R. 

Definition 2.3.2. A family of support structures H on retinas R is said to be highly 
interconnected if, given any positive integer B and positive real number €, one has that if n is 
sufficiently large with respect to B, then for any subset A c R with \A\ > €n, either ' 
covjj(A) > B or else |Con u(A)\ > {l-t)n. A predicate / is said to be highly interconnected if 
any family of suppor.t structures which admits / on large retinas must be highly 
interconnected. 

Proposition 2.3.3. Any family of support structure* which admits arbitrary predicates must be 
highly interconnected* More precisely, if H is a support structure which admits arbitrary 
predicates on a retina R, then, for any A C R, either covu(A) > \A\ or else 

\Con H (A)\>n-\og 2 (cov n (A)) 

This will follow from another simple counting argument: 



Local and Global in Computation — September 20, 1977 — 12 



Lemma 23 A, Given collection* of Boolean variables X ~ {x* f . . ., x ) and Y « (y», . . ., y.), let 
t be any integer such that I < a and log«l < b. Then there exist functions /: X X Y -* {0 f I) 
which cannot be represented in the form 

fiX t Y)^g(f l (X) f ... f f i (X) f Y) 

Proof. Functions /:XxF> {0, 1} are equivalent to functions /* from X into {0, i} , the set 
of Boolean functions from Y into (0, I}, via the correspondence [/*(X)](F) « /(X, F) 
Representing / in the required form is equivalent to finding a factorization of /*: 




where F is a set of t Boolean variables; *(X) - </|<X) f/ . ., / f (X)) and [^(F)]<F) - ^(F,F). 
To demonstrate that not all functions factor in this way, we need only consider the sizes of 

the sets involved: An element of X can take on 2 possible values, and there are 2r possible 

Y * 

values for an element of {0, i} . To construct a function / which does not factor, first 

enumerate the elements of X: 

v 

and the elements of {0, 1} i 

2r 
There are two cases, depending on the relative sizes of a and b: 

■ . 1. If 2 b > a > t let /%.) « £. for * • I, . . ., 2 a . Then the image of /* contains 2 a 

distinct elements, so that /* cannot factor through F, which has size 2 l < 2 a . 

2. If a > 2 b > 4 let /*(».) - £. for i « J, . . ., 2 r . Then the image of /* contains 2* 

distinct elements, so that /* cannot factor through F, which has size 2* < 2 2 . D 
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Proof of 2.3.3. For any A c R define R(A) c R to be the set (R-Cor\{A)) U A, i.e., delete from R 
all variables which Me in Con(/l) but not in A. The inclusion of R{A) tn R induces a surjection 
{0, 1} R -> {0, 1} W>1) . Set t - cov(/l), X to be the variables in A, and T to be the n-|Con(/I)| 
variables in R(A)-A. Then any predicate on R which is admitted by H induces a predicate on 
R(A) of the form *(/|<J0i / 2 <X), . . ., / f <X), Y). Now apply 2.3.4. D 

In the case of 1/M-fractional supports, the following combinatorial argument shows 
that interconnectedness implies high covering multiplicity: 

Ltmma 235. If II is any family of highly interconnected 1 f M- fractional support structures 
then Km «p (g(H) J> M. Consequently, a highly interconnected predicate cannot have finite 
multiplicity for all fractional supports. 

Proof. First note that if A c R with cov(A) < M-i then \Con(A)\ < n{M-l)/M. So if the family 
of support structures is highly interconnected, we can seiect any number € and be sure that, 
for R large enough, we have cov(A) < M-l implies \A\ < in. 

For any of the retinas ft in the family, let R^ be the subset of R consisting of points 
lying in fewer than M supports: 

R M m {ieR: cov({»}) < Af } 
.hiote, in particular, that f?| consists of all points with covering multiplicity 0. 

Then ^'heRu^W + ^eR-Ru 00 *^ 

But 2 i€R cov({i}) > \R M \ - \R t \ 

and ^ieR-R C0V U»}> > M(n-\R M \) 

nT 

Therefore, we have /*n > Mn - (M-l)\R^\ - |Kj| 



or 



II > M - {M-l)\R M \ln - (IKjI/n) (1) 
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If V c {SA is some collection of the supports in //, define R ff to be the subset of R 
consisting of those points which are contained in only those supports which lie in <r: 

R ff - { ie R: i <£ Sj for S. £ <r } 
Consider now the union of the R ff over all sets <r of M-l partial predicates. This union im- 
precisely the subset of R consisting of points i for which cov({*}) < M-l. That is, 

«Jf - U |<r|-Jtf-1 R <r < 2) 

If r is the rank of H then the number of sets <r containing precisely M*l supports is equal to 

the binomial coefficient ( M _ A Notice that for a given <r, covtR^) cannot be greater than 

|ir|. Thus, according to the remark noted at the beginning of the proof, we have \R^\ < in on 

large retinas whenever |<r| - Jf-I. Equation (2) therefore implies that 

IRj^I < ( A /_ i) 

Combining this with |R|| < trt and equation (1) yields: 

M**M*-I> •(/_■,)--• 

Since the order of H is equal to n/Af, we have that r - jiAf. Substituting this into the above 
inequality and using the fact that f jjj _ j) < 2^ gives 

|i>M"-«[W-J)^ M + J]" (3) 

Now, given any J > 0, choose e small enough so that i [ (M-I)2 +-I ] < & and consider 
retinas with n large enough so that (3) holds. Suppose, for n this large, we had fi < Af-$. 
Then 2* M < 2 M , so «[(M-J)/ M +1] < i, and 

p .♦ < [(M-l) 2* M + I] < M + * < W " * + * < *f 
which would contradict (3). Therefore we must have fi > M-6> D 
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2.4. Interconnectedness of the Connectivity Predicate 



Theorem 2.4.1. The predicate fX is connected! is highly interconnected. 



The proof arises as a generalization of the simple observation that connectivity cannot be 
admitted by a support structure of rank 2 in which the retina is partitioned into disjoint 
halves, as can be seen immediately by considering the connectivity of the figures formed by 
the various combinations S f U T. shown in Figure 5. This construction has been generalized 
by Papert [9] to show that, if we allow only those structures in which no support intersects 
both the left and right halves of the retina, then the rank required for determining 
connectivity must grow arbitrarily large on large retinas: 

Proposition 2.4.2. Suppose II is a family of support structures which admits TX is connected!, 
and that no support in H intersects both the left and right halves of the retina. Then, as n 
increases, the rank of H must grow at least as rapidly as Vn. 

Proof. Consider the family of figures illustrated in Figure 6, each consisting of a square, with m 
horizontal lines meeting the sides of the square at contact points *j, — ,t m , yj, . . ., y m . 
Each pair of Boolean m-tuples X « (*j, . . ., * m ) and Y « (yj, . . ., y m ) gives a figure F(X, Y) ; 
and it is easy to see that one of these figures is connected if and only if each horizontal line 
is connected to the square, either on the left or on the right. In other words, F(X, Y) is 
connected if and only if XVT. (Recall that XVF, "X or Y~ means that for each i we have 

*|Vy,.)' 

Let /,, . . ., / be those partial predicates whose support lies in the left half of the 
retina. Then, for any figure F - F<X, K), the o-tupie f L (F) - (f^F), . . ., jf a (F» depends only 
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Figure 5 

A simple rank two structure cannot detect connectivity 

for the figures S.gT, 
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Figure 6 
Connectivity figures for Proposition 2.4.2 
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on the m-tuple X, i.e., / L (F(X, Y)) - ftfX). We claim, therefore, that a > m. For, if not, then 
there are two distinct m-tuples X { and X 2 with ftfXj) - / t (Xj). But then, for any m-tupie- 
Y, we have 'F(XpY) is connected if and only if F(X 2 > Y) is connected. On the other hand, 
taking Y - ~Xj and noting that F(Xj ~Xj) is connected, shows that we must have X^V^Xj. 
Similarly, taking Y - ~X 2 gives XjV^X^ Therefore Xj - X^, and so a £ m . Finally, observe 
that on a retina of size n, we can chpose the number of horizontal strips to be proportional to 

v*. □ ■■ 

The next step in the proof of 2A1 is to extend the above construction so that the 
"contact points" x. and y. can be distributed throughout the retina: 

Lemma 2.4.3. For any integer m there i* a constant K(m) melt that if H admit* connectivity on 
a retina R, and A C R with \A\ > KJn 9 then either cov(A)> m or else \Con(A)\ > R - KVn. 

Proof. Let R, c R be the subset of R consisting of all points which do not lie within distance 
2m of the boundary of R. Choose points Xj, . . ., x m lying in A such that (1) each x. is 
contained in Rj (2) For i t j, x i and x . do not lie within the same horizontal row of R, and, 
moreover, the horizontal rows containing x. and x . are at least six squares apart. (See Figure 
7.) Notice that we can do this so long as \A\ is greater than |Rj| plus the size of 6m horizontal 
strips, i.e., \A\ > 14V n. Next, choose points yj, . . ., y m in R-Con(») such that (1) each y . lies 
in Rj (2) the horizontal row containing any v - lies at least 6 units from the row containing any 
y^ (jtj* *) or any x^. We can do this so long as |R-Con(>1)| > IBm-Jn* To prove the Lemma we 
show that these x's and y's can be used as "contacts" in a family of figures equivalent to the 
figures of 2.4.2. This will imply at once that cov(/l) > m. 

We construct the figures as follows: Begin by renumbering the x^s from top to bottom, 




Figure 7 



Choosing the points x, in Lemma 2.4.3 



S^ Local and Global in Computation — September 20, 1977 — 17 



i.e., so that x» is above x - for i<j. Next arrange each r. to be a pontact point for three 
horizontal "wires" as shown in Figure 8. Extend each of these wires on the left to meet the 
boundary of R*. Now do the same thing for the y% only this time working to the right. Next, 
in the boundary of /?•, connect the bottom wire of each sn to the top wire of arj+jt and 
similarly for the ys. Connect the top wires for x. and y., and the bottom wires for x m and 
y m ^ na 'fy> make- a connection "around the bottom of R" to join the middle wire of ar. to the 
middle wire of the corresponding y», as follows: for (x ., yfl work in the boundary of K; for 
(*r«>yn), in the strip 3 units in from the boundary; and so on. These latter connections all lie 
in R-Rp and so do not interfere with the previous wires. (The final figure is illustrated in 
Figure 9 for the case m « 3.) As in 2A2, these are connected if and only if XVY. D 

Proof of 2.4.1. Given M and €, let K be the constant K(Af+l) given by Lemma 2A3, and choose 
R large enough so that KVn < «n. Then, if \A\ > tn and |Con(/l)| < (l-*)h , we have that \A\ and 
|K-Con(>1)| are both greater than KJtu Hence cov(A) > M+I by 2A3, which shows that the 
computation is highly interconnected. D 

2.5 Covering multiplicity of the connectivity predicate 

Applying Lemma 2.3.5 to Theorem 2A1, we see immediately that connectivity, unlike 
the "local" predicates for local convexity and IX is a single solid rectangle! discussed in 
Section 2.2, does not have finite covering multiplicity over all fractional supports. In 
particular, I/Af-fractional support schemes which compute connectivity on large retinas must 
have Nro^oof* > M* But connectivity, we expect, should be "even more global" than that — 
more like computing arbitrary Boolean functions, where, for any fixed M, we have 
l' m n -»coM ■ °° by 2.2.3. Indeed, the same reasoning as in 2A2 shows that this must be true in 
connectivity computations, and that fg must grow as rapidly as Vn, so long as we assume that 
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Figure 8 



Details of the contact x. in Lemma 2.4.3 
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Figure 9 



Connectivity figure for Lemma 2.4.3 
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the supports of the partial predicates are disjoint. In this section we show that connectivity 
has unbounded covering multiplicity for I/Af-fractional supports, even if the supports are 
allowed to overlap. We shall not, however, consider the case of arbitrary overlap. Rather, we 
restrict attention to schemes in which the supports overlap "regularly" according to the 
following prescription: 

Definition 2.5.1. Let R be a square retina which is partitioned into D square blocks of equal 
size. Then a d/D-regular mpport structure is a structure in which each support consists of 
some fixed number d<D of these blocks. 

Note that this is a special case of d/D-fractional supports. Supports may overlap, but they 
must overlap "regularly" — two intersecting supports must necessarily have an entire block in 
common. We remark also that, although we phrase our results here only for square retinas, 
the same sort of things will be true for all families of retinas with a sufficiently large 
"interior," i.e., for sequences of retirvas R in which the perimeter of R grows no faster than 
V|R|. 

Theorem 2.5.2, For any family of d/D-regular support structures which computes connectivity, 
the covering multiplicity must satisfy Hm^^ji « «o. 

The proof uses the switching network construction to translate the problem of determining 
connectivity into that of computing an arbitrary Boolean function. Recall [7] that if 
X«(*j, . . ., x a ) is a set of Boolean variables, and /: X -» {0,1} is a predicate, then a switching 
network for f is given by a function F which associates to each Boolean c-tupie a figure on a 
retina R such that F(x^ . . ., r a ) is connected if and only if /(*j, . . ., % a ) - J. We can 
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construct such a network by writing /in conjunctive normal form and translating the Boolean 
expression into a network, interpreting conjunction as series coupling and disjunction as 
parallel coupling. For example, the predicate 

fix,, x 2 , x 3 ) - Tat least 2 of the variables are equal to 11 
has conjunctive normal form 

(x l yx 2 yx 3 ) a (^x l \ix 2 yx 3 ) a (x l y^x 2 Vx 3 ) a (x^x^x^ 

and the corresponding network is shown in Figure 10. The figure is interpreted as follows: 
when the variable x- is equal to I, the squares marked "xf are filled in and the squares 
marked "~xj* are left empty. Conversely, when *j - 0, the squares marked m ~xf are filled in 
and the squares marked "x" are left empty. These squares are the called the contacts of the 
network. 

Notice that a function of a variables can have at most a2 a terms in its conjunctive 
normal form, and hence a network with this many contacts can realize any Boolean function of 
c variables. This is the sense in which computations for determining connectivity must also be 
able to compute arbitrary Boolean functions. The proof of 2.5.2 now rests on the following: 

Lemma 2.5.3. Suppose tee have D sets of Boolean variables 

; ■ •" Xj - ix u , . . ., x lm ), . . ., x D - {x DV . . ., x Dm ) . 

Consider support structures of order dm in which each support consists of some collection of 
d < D of the X •• Then for any fixed values of d and D, and any bound B, toe can choose m 
large enough so that there are Boolean functions of the X*s which cannot be computed by 
structures of this type having covering multiplicity fi < B. 

This lemma follows at once from 2.2.3. Moreover, by examining the proof of 2.2.3, we see that 
li would have to grow large with the the same order asm, D 



/•"-"N 




Figure 10 
Switching network for 



(xV x 2 v x 3 ) A -(is/XjV x 2 v x 3 ) A(x i -V-'x 2 v x^ a (x^ ^v^) 
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Proof of 2.5.2. Construct in R a switching network that can realize any function of Dm 
variables, such that the contacts corresponding to the variables in the ith set X* all lie within 
the »th block. Then any regular support structure which admits connectivity on R must also 
admit arbitrary Boolean functions as in 2.5.3. As R becomes large we can choose larger and 
larger values for m. Therefore p must also increase without bound. □ 

Remarks. The above proof shows that p must grow at least as rapidly as log n. This 
logarithmic factor arises from the use of the general switching network to realize arbitrary 
Boolean functions. It is natural to ask if there is a bound more in keeping with the Vn growth 
observed for disjoint supports. Also, is there some way to eliminate the "regularity" 1 
assumption, and so establish 2.5.2 for arbitrary l/M -fractional supports? What is the order of 
growth in this case? 

3. Local and global in real-valued computations 

^he techniques presented above for analyzing Boolean functions are also applicable to 
the study of real-valued functions. The "retina" in this setting is an index set for n real 
variables {x^ . . ., x n \ and the support of a function /: lR n -t R consists of all indices j such 
that / depends on x^. This extension to the real-valued domain is analogous to Uesaka's 
extension of Minsky and Papert's work to the theory of "analog perceptrons" [10], [11]. The 
analog perceptron formulation, however, deals only with functions of the form 
JF(xj, . . .,~* n ) ■ 2^/$ where each /• is a function of (hopefully) fewer than n of the x's. In 
keeping with the comments in Section 1.2, we find that this linearity requirement is too 
restrictive to serve as a basis for a general study of structural complexity. For example, the 
function 
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mult: (at j,. v ., * n ) -» Xj ' * * * n 
cannot be written as a sum of functions /. of fewer than n of the r's [10] and is therefore 
"global" from the perceptron point of view; but allowing multiplication, rather than merely 
summation, at the "output stage" would enable us to compute mult using any partition of the 
(xt f . . ., a: ) as supports for the f*. (This is exactly analogous to the comments on the parity 
predicate in Section 1.2.) 

In the sections below we shall work in the category of real-valued differentiate 
functions. From this point of view we analyze, in Section 3.2, the covering multiplicity 
necessary compute arbitrary polynomials. Section 3,3 applies the same ideas to computing the 
determinant of a matrix and solving systems of linear equations. All of these results are based 
on a theorem on functional decomposition proved in Section 3.1. 

3.1. The Decomposition Theorem 

Following the framework of Section 2 we will say that a support structure <Sj, . . ., S r ) 
admits diffcrentiably a function /: JR n ■ -* R if / can be represented in the form 
/ - #(//> • • •> f r ) where support(/.) c Sj and g, /j, . . ., f f are differentiable. We will also 
consider situations in which / is defined only locally in some neighborhood I/c R n in which 
case we require that g and the /. be locally defined. To investigate conditions under which 
this can be done, consider first a simple kind of support structure in which the variables x* 
are partitioned into two disjoint sets X - (x v .. . ., x a ) and Y - (* +j, . . ., * n l Let Diff(X) 
denote the algebra of real-valued differentiable functions of the variables in X. For any 
/: R n -> IR let A(X, /) denote the module generated over Diff(X) by the a functions 
bf/bx lf ...,bffbx ar 
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3.1.1. Decomposition Theorem. // /: V C R* 2 X R n ~ tt -* R U a differentiable function which can 
be represented as 

flX 9 Y)-'g(f l (X),..«f r iX)fY) 
where gi |R r+ ^~ a > --+ R and fy R a -> R, i - I, . . , r ore differentiable, then the module A<X, /) 
ha* rartfc at mojit r throughout CA Conversely, i/ / i* corttirmouiiy differentiable^ and A(X, jf) 
has rank at most r, then there is an open subset V C U on which f can be so represented. 

Proof. Differentiating the equation / - g(fp . . ., f^ Y) with respect to any x . € X gives 

bf/bx.^ljibg/bf^ibfj/bx.) 
Each fp and therefore each bf*fbx* t lies in Diff(X). Hence bffbx. lies in the module 
generated over Dif f(X) by the r functions bgfbx*. Since this is true for all t, we have that all 
of A(X, f) is contained within this module and is therefore of rank at most r. 

To show that the condition Is also sufficient, begin by choosing any p - (py, py) € U 
and r+1 sets of particular values Y j, . . ., Y r+ j for the variables Y such that the pairs (py, py) 
all lie in U 9 and consider the functions f*(X) - /(X, K.). We claim that for any values Y . the 
matrix 

HVi/^/M i - I, ..., r+1; /- I, ..., a 
has rank at most r. 

To prove this, pick any r+1 columns, say, for notational simplicity, columns I through 

r+L According to the hypothesis on A(X, /) we have 

bf(X, Y)lbx r ^ « X jml ^ r <fi r + u (X)(bf{X,Y)/bxj) 

where the ^ r+ j y(X) are functions of X alone. Thus we can substitute any Y^ for Y m the 

above equation to get 

Since the ^ r+J .(X) are independent of I 'this shows that . 



Local and Global in Computation — September 20, 1977 — 23 






((r+i)st column of matrix) - X • ^ r+ j .(X) (/th column of matrix) 
which proves the claim. 

Now to complete the proof of the theorem, choose Fj, . . ., K j to maximize the rank 
of ||d/j/d*y||. Denoting this maximum rank by m, m < r by the claim. Without loss of 
generality we may assume that the first m rows of the matrix are linearly independent. 
Consider now the function 

F:t/->K m+i xR n ' B . 
where 

F(X, Y) - (f(X, 7), /,(X), . . ., f : m (X), Y) 

By choice of the Fj, , F f+ j we have that the rank of F is at most tn+n-eu Therefore the 

functions describing the image of F are functionally dependent [1J (This is where one needs 
that /is continuously differentiable.) Moreover, the set of points at which the /•,...,/ 
have maximal rank forms an open set' K c V t and at any such point p - (py, py) € V we can 
apply the Implicit Function Theorem to solve the functional dependence relation for /(X, F), 
i.e., there exists a continuously differentiable function g defined in a neighborhood of py such 
that "' 

/TX, F) - gif^XYt . . ., / m (X), Y) D 

Note: In the case r-1 this theorem reduces to a result of Leontief [4]. 

To relate 3.1.1 to the framework of Section 2, we define Drff(fl) for any B c {1, . . ., n} 
to be the algebra of differentiable functions in all the variables. (*.)• ^ « . 



^■"*\ 



Local and Global in Computation — September 20, 1977 — 24 



Corollary 3.1.2. Suppose // is a support structure which admits differentiably the function 
fx IR n -♦ R, and A C {!,..., n}. Then among the a derivatives (bf/bx^ € «, at most cov *,(/!) 
are linearly independent over Diff(Con» A A)). 



Prpof. Proceed just as above, noting that, for any i e A 

bf/bx.-Xjibgfbfpibfjfbxi) 
where the summation ranges over / € Cov({»})» and that for any such j 

bfj/bx. € Diff(Con({i})) c DifftConOW. D 

3J2. Polynomial evaluation 

We now apply the results of 3.1 to prove that support structures which admit 
arbitrary polynomials must be highly interconnected, and of infinite covering multiplicity for 
fractional supports: 

Proposition 3.2.1. // H is a support structure which admits differentiably the polynomial 
P(x v . . ., x) - 2J. £ : XsxJ then for any A C {], . . ., n}, either cov(A) > \A\ or else Con(A) is 
the entire index set {I, , rt}. 

Proof. Suppose that there is some index i not contained in Cor\(A). By renumbering the r's we 
may assume that this index is n, and that A « {I, . . ., \A\}, {This renumbering does not change 
P, which is symmetric in the x>.) Then Diff{Con(/l)) c Difffxj, . . ., * n _j). Moreover 
P(* J( • • ., *„) - \j xjrj - I y XfJ * X„ n Vj XjxJ 
' *n" + *jtn *?J * 2^ n *„*." modulo Diff(* J( . . -, *„_,) 

and so, taking the derivative with respect to xiiktn) gives 

bP/bx k - x n k + nx n a: fc n " i modulo Difftej, . . ., ar^j) 
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As the reader can verify, this equation implies that the module generated by (dP/d*. )r « 
contains the \A\ elements x n , r R , . .. ., x^ \ which are linearly independent over 
Dif f(* j, . . ., * n _j) and hence over Diff(Con(/I)). Therefore, by 3.L2 we have cov(A) >\A\. D 

Corollary 3.2.2. Any family of tup port structures which differentiabty admits arbitrary 
polynomials must be highly interconnected. Consequently in any family of such structures 
with 1 1 M- fractional supports the covering multiplicity satisfies lim ^ fi > M. 

This follows at once from 3.2.1 and 2.3.5. D 



3 Ji Matrices and linear equations 

We turn next from the computation of arbitrary polynomials to consider the specific 
polynomial which expresses the determinant of a matrix in terms of the matrix elements. The 
retina here is an index set of n real variables (x^, . . ., x nn ) which we view as a matrix 
X - ||*y||. The function det: (*. .) -* det(X) is a polynomial of degree n. Expanding det by the 



Ith row shows that 



ibfbxij) det - ± X 1 ' 



where X 1 * is the cofactor of x*. t itself a determinant of order n-1. 



Lemma 3.3.1. Let S - (sp . . ., * n ) be any collection of n of the variables x . .. (That is, each s» 
represents some x*..) Let X k be the cofactor of # fc . Then the cefaclors X* t . . ., X n are 
linearly independent over D\tf(sp ..., s n ). 

Proof. By permuting the rows and columns of X and reordering the *. 's we may assume that 
*1 m X -1V ^ e c °f ac tors X , . . ., X are sums of monomials of degree rt-2. The key to the 
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Lemma is the claim that, in the expansion of X li there is at least one monomial which does not 
contain any of the r's lying in S. 

To prove this claim we note that, since X U is itself a determinant of order rt-2, the 
monomials a in the expansion for X 11 *re the products a - a, . . . a., where a - lies in the 

..'.. __ ; __ _ ......".„.■ _ ._ _ - -• - ' - - •;•— • • ft..: _.. ,J 

/th column of X and no two a/s lie in the same row. So we must show that there is at least 
one such set of ay's, none of which lie in S. Let c . be the number of elements of S lying in 
the /th column of X. By permuting the columns, we may assume that c*>c 9 > .'. . >c . There 
are two cases to consider: 

Case 1: cj > I. We want to choose a 2 from the second column. There are rt-i-c« possible 
choices. Likewise, in choosing a 3 from the third column, and not lying in the same row as 
dp there are n-2-c 3 choices. In general, there are rt-(/-!)-c . choices for a... We need to 
know then, that 

n> C j + (H) /-2,...,n (4) 

in order to be sure that there are choices possible at each step. But 2c - n and 
c 2>^> . . . >e n so that Cy < n/j. Also, if Cj > 1 we can be sure that c .'« for / > n/2. So 
we need only verify (4) for j < m 

cj + (/-I) < (n/j) + (j-l) - [y-«)(/-I)/y].+ "n <"n 
Therefore we can construct a as required. 

Cas0'2:.C| - i. In this case each column of X contains one element of S. Now, if some row 
contains more than one element of S we can prove the result by applying Case 1 to the 
transpose of X (which has the same determinant as X). Otherwise, if each row and column 
contains one element of S, we may permute rows and columns so that the s. are precisely the 
diagonal elements of X. If so we can take the monomial a to be, e.g., X 9 ,X*/ ' • X ,X , .. 
This proves the claim. 

Now to complete the proof of the lemma, let F - Difffc,, . . ., * n ), and suppose that the 
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X were linearly dependent over F, i.e., 

^k ?k X m ° where f k € F 

Now let a be the monomial in X 1 whose existence is assured by the above claim, and set 
X ~ a+0. We would then have 

h«--ffi-*k-2„ n fk xk (5) 

Let t lt ...,t^ be those * y which do not lie in 5 and regard (5) as an equation in the 
polynomial ring F[t Jt . . ., t 2 J. The construction of a guarantees that, as an element of this 
ring, o has degree n-J. Also, the terms on the right side of (5) have degree at most n-1. 
Moreover, none of these terms can be equal to «, since a monomial in the expansion of the 
cofactor of an element of a matrix cannot also lie in the expansion of a cofactor of a different 
element of the matrix. Therefore equation (5) is impossible. D 

As a consequence we deduce a theorem about the distributed computation of 
determinants: Suppose we have an nxn matrix X, whose entries we partition into n sets S- of 
order it. (The S f can be the rows, columns, SnxVn submatrices or whatever.) Consider 
computing dettf) by first evaluating separate "preprocessing" functions /. of the S. and then' 
combining the results of the /. by some function g. We find that computing a determinant is a 
worst case for this kind of distributed computation: 

Theorem 3.3.2. Ut H - (S,, . . ., S„) be a support structure for function, of n 2 variables tvhere 
the Sj are pairwise disjoint and of size n. If H differentiably admits the determinant 
function, then cov(S .) > n for all /. 

Proof. Since the Sj are disjoint, we have Con(S y ) - S. and the theorem follows immediately 
from 3.3.1 and 3.1.2. □ 
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In other words, no matter what "local" computations we make based on the n elements of each 
Sj, we still must transmit n numbers to be combined by g. From the communication point of 
view, there is no point in doing any local computations at all. We may as well transmit the n 
elements of Sg to g directly, 

As a final application of these ideas, consider the problem of solving an rtxn system of 
linear equations Xy « b based on dividing the matrix X into columns. That is to say, each of 
the elements y. should be computed by a structure in which each support can intersect at 
most one column of X (and we'll also allow each partial function to access any entry of 6). As 
with determinants, this turns out to be a worst case: the information needed from each 
column of X cannot be transmitted by fewer than n differentiabie functions: 

Theorem 3.3.3. Let b be a fixed non-zero n-vector> and let yAX) be the j-th entry of the 
solution to the system of linear equations Xy « b. Suppose that H is a support structure 
which diffcrentiably admits y ., and that each support in H is contained tcithin a single 
column of X. Then, for the j-th column X - of X, we have cov(X .) > n. 

Proof. Let b J X denote the "augmented matrix" in which the /th column of X is replaced by 6. 
By Cramer's Rule, y. - \b*X\l\X\. Hence, for any element x- k 

z>yj/z* ik - i/\x\ 2 [ (ixwb^'xybx^) - (|6^ixa|x|/a* tt ) ] 

For any /, if b is non-zero, there are matrices. X for which |X| if and \b J 'x\ yt 0. So consider 
the elements in the /th column of X, i.e., take / - k in the above equation. Since \b*X\ does 
not involve the variable *.., b\b J 'x\fbx. . - and so 

a V & *0* - - (l^'xi/ixi 2 ) aixi/a* y - ± (\bh\f\x\ 2 )x^ 

But, by Lemma 3.3.1 the X lj \ X 2 *, . . ., X n ' are linearly independent over Diff(r J .,..., x n .). 
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Hence the multiples of these polynomials by \b J X\/\X\ 2 are also linearly independent. 



Corollary 3.1.2 therefore implies that covQC .) > n. D 



3.4 Relations with Hilbert*$ 13th problem 

The preceding perspective on the complexity of functions is related to investigations 
growing out of Hubert's 13th problem, which concerns the possibility of representing functions 
of several variables as superpositions of functions of a smaller number of variables. (For a 
survey see [5j) Notable among these is the result due to Kolmogorov and ArnoPd [3] that 
any continuous function can always be expressed as a superpostion of continuous functions of 
two variables. On the other hand, it is known that this cannot be done if the functions in the 
decomposition are required to satisfy differentiability constraints. This qualitative difference 
between differentiable and non-differentiable decompositions suggests there is no 
straightforward extension of the techniques of the previous sections to allow for 
non-differentiable partial functions. 
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4. Conclusion; Questions for further research 

This paper has suggested precise computational formulations which interpret such 
vague notions as "global," "gestalt" or "the difficulty of dividing a computation into 
independent simpler parts." The goal is to develop meaningful measures of complexity which 
reflect only how the pieces of a computation are interrelated, and are independent of the 
specific operations performed by each piece. Hopefully, such an effort could lead to a unified 
perspective for discussing problems of "parts and wholes" in computational geometry, in 
numerical computation, in distributed data processing, and perhaps even in artificial 
intelligence and cognitive theory as indicated in [8\ 

The theory is still at an embryonic stage, and the reader will no doubt recognize 
numerous ways in which the above results can be improved. What can one say about the 
order of growth of covering multiplicity for connectivity or other geometric predicates? Are 
there general techniques for establishing lower bounds for the covering multiplicity of specific 
Boolean functions? The Decomposition Theorem of Section 3.1 is a differentiate analogue of 
results of Ashenhurst on "disjoint decompositions" in switching theory [2]. Can this be 
extended to more general decompositions? We have only hinted at applications to the study 
of distributed data bases. Developing covering multiplicity criteria in this context is surely a 
major area left untouched by the present investigation. Other extensions of the theory could 
deal with "continuous retinas," in which the subsets S cR become measures defined on the 
plane, and the predicates /become functions on the Hilbert space of measures. It is also 
important to develop alternative measures of "local and global" complexity and contrast these 
with covering multiplicity. For example, there should be a whole spectrum of .. 
"interconnectedness" running from non-overlapping support structures to the highly 
interconnected structures defined in Section 2.3, and one should be able to measure precisely 
the "intrinsic interconnectedness" required for computing a function. Certainly, much remains 
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to be done in this area. 
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